This application is based upon and claims the benefit of priority from the prior Japanese Patent Applications No. 11-334295, filed Nov. 25, 1999; and No. 11-336286, filed Nov. 26, 1999, the entire contents of which are incorporated herein by reference.
The present invention relates to similar image retrieving apparatuses, and more particularly to a similar image retrieving apparatus for retrieving similar images from image database with respect to three-dimensional medical use images such as an MRI, a CT or the like, and complicated three-dimensional shape image data or three-dimensional CAD data or the like which can be obtained from the three-dimensional measuring device or the like.
Furthermore, the present invention relates to three-dimensional database apparatuses and a methods for constructing three-dimensional image database, and more particularly to a method and an apparatus for extracting region of interest and calculating a feature quantity from the three-dimensional image, thereby constructing database for retrieving similar images on the basis of these data items.
Conventional examples of a system for retrieving similar case images associated with similarity of lesion portions with respect to medical images such as lungs include a method described in Literature 1: xe2x80x9cLocal Versus Global Features for Content-Based Image Retrievalxe2x80x9d by C. R. Shyu, C. E. Brodley, A. C. Kak, A. Kosaka, A. Aisen and L. Broderick, in pages 30-34 of the Proceedings of IEEE Workshop on Content-Based Access of Image and Video Libraries, Santa Barbara, Calif., Jun. 1998.
In this method of literature 1, the region of interest desired by a doctor is set by the masking of the lesion portion localized within the CT image of the lung by the doctor who is the user of the method, so that the case images similar to the masked region are retrieved from the medical use image database.
There is available an advantage in that more appropriate retrieval is enabled with the method of literature 1 as compared with conventional methods in order to accurately extract the lesion portion intended by the doctor who is the user of the method.
Similar image retrieving systems corresponding to three-dimensional models include a system described in literature 2:xe2x80x9cShape-Similarity-Based Retrieval in Image Databases by R. Mehrotra and J. Gary, in pages 55-86 of Chapter 3 of Image Description and Retrieval, edited by E. Vicario, Plenum Press, New York, 1998.
The method of literature 2 in this paper is a method for calculating a similarity of polygons by registering a three-dimensional model of the object as a CAD model of a linear image and using a position relationships of the linear image. The method has an advantage in that a three-dimensional object can be retrieved which can be represented in the CAD or the like.
In the method proposed in literature 1 described above, the image to be used by the doctor is basically a two-dimensional image. The masking of the lesion portion is merely two-dimensional image as well.
However, the lesion portion treated in the inspection image such as the CT, the MRI or the like is distributed in a three-dimensional manner. It is difficult to accurately describe the three-dimensional distribution and the three-dimensional position of the lesion portion in an accurate manner.
For example, in the CT and the MRI images in the brain surgery, it is thought that the three-dimensional distribution of the lesion portion and the position thereof have a significant meaning. There is a disadvantage in that it is difficult to accurately retrieve the case images only with the method of literature 1.
Furthermore, in the method proposed in literature 2 described in literature 2, the three-dimensional object is limited to an object that can be described in a simple polygon model. The object is described with the end points and line portions of such polygons, and only the position relations thereof are used. The method does not include the modeling of the texture peculiar to the three-dimensional object and the modeling of the position relationships of the three-dimensional objects.
Furthermore, conventionally as a system for retrieving images, a system using a two-dimensional image data is generally used.
These systems extract features from two-dimensional image data, accumulate the features quantities in the database and retrieve the images on the basis of these features quantities.
Furthermore, in recent years, along with the prevalence of the three-dimensional image, only regions in which users take interest are extracted from the three-dimensional data and the regions are transformed into three-dimensional object to be displayed on a display device with the result that each kind of inspection is conducted more and more frequently in recent years.
However, this three-dimensional image data is extremely large in quantity, and the usage thereof is extremely complicated.
At the time of accumulating the large quantity of this three-dimensional image data in the database, it is necessary to efficiently construct the database itself in the form in which the database itself can be retrieved with ease later in order to enable the user to efficiently retrieve the image data demanded by the user out of this image data.
In order to attain such purpose, Jpn. Pat. Appln. KOKAI Publication No. 5-159001 discloses an apparatus for retrieving three-dimensional figures.
This apparatus is such that two-dimensional information such as front views, sectional views, bird""s eye views or the like which represents well the feature of the figures is accumulated in the database with an addition of retrieval keys to the figures, so that upon the figure desired by the user being retrieved, information on the corresponding three-dimensional figure is detected while subsequently displaying the associated two-dimensional information on the screen.
Furthermore, in a three-dimensional image processing apparatus according to Jpn. Pat. Appln. KOKAI Publication No. 6-215105, there is disclosed an example in which three-dimensional geometric shape information and attribute information of an object is stored, and images are retrieved on the basis of this three-dimensional geometric shape information and the attribute information with the result that the two-dimensional projection images are displayed with the three-dimensional geometric shape information and the attribute information given as a result of image retrieval.
However, the two-dimensional image retrieving system as the prior art described above has a problem in that the feature used in the retrieval of the images is obtained from the two-dimensional image data, and the feature quantity inevitably accumulated in the database becomes two-dimensional, and the feature-of the images having three-dimensional-like information cannot be represented, so that the system cannot be used as means for retrieving three-dimensional-like similar image data.
Furthermore, in a system for retrieving three-dimensional figures disclosed in Jpn. Pat. Appln. KOKAI Publication No. 5-159001 described as the prior art described above, so-called structure data such as CAD data or the like is primarily retrieved, and only wire-frame-like information can be accumulated in the database with respect to such structure data.
However, general three-dimensional images have not only such structure data but also complicated data such as information on the pixel contrast, texture information or the like.
Consequently, there is a problem in that the apparatus for retrieving three-dimensional figures disclosed in Jpn. Pat. Appln. KOKAI Publication No. 5-159001 cannot retrieve the figures in consideration of the fine shape changes on the surface which can be assumed from the features of complicated information inherent in general three-dimensional images and in consideration of similarity on the inside information of objects.
Furthermore, this apparatus provides only a plurality of displays of two-dimensional information with respect to the display of the result of retrieval. However, in order to display three-dimensional images on the display device, it is difficult to accurately transmit the image information to users unless the display is provided from an arbitrary viewpoint. From this viewpoint, there arises a problem in that detailed three-dimensional information cannot be displayed visually.
In the same manner, in the so-called three-dimensional image processing apparatus disclosed in Jpn. Pat. Appln. KOKAI Publication No. 6-215105 as well, the input information is only the position of the tips of polygon patches, the connection information of the tips, and the material characteristic, and texture features inherent in objects are not taken into consideration.
An object of the present invention is to provide an similar image retrieving apparatus which can be used in three-dimensional medical use images such as MRI""s and CT""s, complicated three-dimensional shape image data or three-dimensional CAD data which can be obtained from a three-dimensional measuring device, or three-dimensional CAD data, the apparatus treating three-dimensional images and objects in consideration of the region of interest and the intention of the user and enabling more accurate retrieval thereof.
Furthermore, another object of the present invention is to provide three-dimensional image database apparatus and a method for constructing three-dimensional image database wherein effective retrieval of three-dimensional images is enabled by using peculiar features inherent in three-dimensional images in order to retrieve the three-dimensional images.
In order to accomplish the above object, the present invention provides a similar image retrieving apparatus comprising:
a region of interest setting portion for setting a portion of a three-dimensional image of retrieved object as a region of interest;
a feature quantity calculation portion for calculating a feature quantity of the region of interest set by the region of interest setting portion;
a similarity calculation portion for calculating a similarity between the retrieved object and three-dimensional image data stored in the image database in advance by comparing the feature quantity of the region of interest of the retrieved object which quantity is calculated with the feature quantity calculation portion with the three-dimensional image data where a region of interest stored in the image database is set in advance and the feature quantity of this region of interest is calculated; and
an image selection portion for selecting similar images in an order of decreasing similarity from the image database on the basis of the similarity calculated with the similarity calculation portion.
The above similarity calculation portion is characterized by calculating the similarity of the image with respect to only the image with which a key word agrees out of the key words selected from a set of key words prepared in advance and images to which key words are added in advance which images are stored in the image database.
Furthermore, the feature quantity calculation portion is characterized by calculating the feature quantity by setting in advance the three-dimensional standard model which is compared in the image retrieval and using a position posture relationships between the standard model and the retrieved object image data.
Furthermore, in order to accomplish the object, the present invention provides a three-dimensional image database apparatus for retrieving an image similar to an input three-dimensional image, the apparatus comprising:
an image input portion for inputting three-dimensional image data;
a region of interest setting portion for setting a region of interest on the basis of the designated condition with respect to the three-dimensional image data input with the image input portion;
a three-dimensional reconstruction portion for reconstructing three-dimensional image data by normalizing and visualizing this region of interest of the three-dimensional image data set with the region of interest setting portion; and
a three-dimensional data accumulation portion for accumulating the three-dimensional image data reconstructed with the three-dimensional reconstruction portion in correspondence to the three-dimensional image data input with the image input portion.
The above three-dimensional image database apparatus has a features calculation portion for calculating feature quantity of the image data with respect to the region of interest of the three-dimensional image data set by the region of interest setting means;
wherein the three-dimensional image data accumulation portion accumulates the feature quantity of the image data calculated with the feature quantity calculation portion in correspondence to the three-dimensional image data input with the image input portion.
Furthermore, in order to accomplish the above object, the present invention provides a method for constructing image database for retrieving image data similar to the input three-dimensional image data, the method comprising the steps of:
inputting three-dimensional image data;
setting a region of interest on the basis of the designated condition with respect to the input three-dimensional image data;
reconstructing the three-dimensional image data by normalizing and visualizing the region of interest of the set three-dimensional data; and
accumulating the reconstructed three-dimensional image data by allowing the image data correspond to the input three-dimensional image data.
Additional objects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objects and advantages of the invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out hereinafter.